1. Field of the Invention
The present invention relates to a digital data encode system and a water mark data inserting method for inserting water mark data in the digital data signal having a series of field data.
2. Description of the Related Art
Recently, there has arisen the necessity of developing means for preventing illegal copying of digital data, especially digital image data, with the advance of an information processor and a communication network and the electronization of various media.
Data encryption technique has been proposed as this kind of technique for preventing illegal copying. This digital data encryption technique is, for example, to enable it to reproduce the encrypted digital image data in only a reproducing system having a proper cryptoanalysis key when digital image data is encrypted. The conventional encryption technique, however, has such a defect that if the encryption code is once broken, it cannot protect against the illegally copying thereafter.
Therefore, a technique by use of water mark data is proposed, as another technique for preventing illegal copying of digital data, free from such a defect as a digital encryption technique has. Water mark data (digital water mark) is the special information to be embedded in digital image data itself in order to prevent the illegal use and copying of the digital image data. It includes, for example, information for authenticating the copyright ownership and judging the infringement of the copyright, and the copy protected information for preventing illegal copying itself.
Such water mark data to be inserted in the digital image data includes two kinds of visible water mark data and invisible water mark data. Visible water mark data means such a special character, symbol, or other data to be inserted in the image that a person who sees the image with the water mark data combined therewith may perceive the water mark visually. This kind of visible water mark data would naturally cause the deterioration of the image quality. While, it is effective in visually appealing protection against illegal copy or illegal data fluid.
A conventional visible water mark data embedding technique is disclosed in, for example, Japanese Patent Publication Laid-Open (Kokai) No. Heisei 8-241403, "Digital Watermarking Free from Image Color Change" (hereinafter, referred to as the conventional technique 1). According to the water mark data inserting method of the conventional technique 1, when combining visible water mark data with the original image data, the water mark data is combined with the original image data in a way of changing only the brightness of the pixel corresponding to the non-transparent part of the water mark data without changing the chromaticities, of all pixels of the original data. At this time, the scaling value for changing the brightness component of a pixel is determined by the value of, for example, chromatic component, random numbers, and pixels of water mark data.
Another example of the conventional visible water mark data embedding technique is disclosed in, for example, Japanese Patent Publication Laid-Open (Kokai) No. Heisei 5-236424, "Information Embedding Method and Its Apparatus" (hereinafter, referred to as the conventional technique 2). The water mark data inserting method according to the conventional technique 2 comprises detecting means for detecting a region meeting a predetermined condition from image data, thereby to embed the water mark in every image data at the position corresponding to the region meeting the condition. The position for embedding the water mark depends on the content of the image. Therefore, it is very difficult to remove the water mark without much deteriorating the image quality.
On the other hand, invisible water mark data means such special data to be inserted in the image that a person who sees the image with the water mark data combined therewith cannot perceive the water mark visually. This kind of invisible water mark data is embedded in the original image data in consideration of causing no natural deterioration of the image quality. As is understood from this, invisible water mark data is preferable to visible water mark data under the condition that a water mark preferably exists outside of the image data to be protected.
The water mark data inserting technique by use of an invisible water mark causes little image deterioration and when embedding special information enabling the identification of a writer as water mark data, it can specify the writer by detecting the water mark data even after illegal copying is performed. By embedding the copy protected information for disapproving copying or the copy prohibited information for prohibiting copying, in the original image data, and providing a reproducing device for reproducing the image data with a special function corresponding to the information, for example, it is possible to notify a user of the reproducing device that the image data is the copy inhibited data and to operate the special function within the reproducing device (copy protection function or the like) so as to restrict copying into VTR (Video Tape Recorder) when the reproducing device detects the copy protected information or the like.
As the technique of embedding the invisible water mark data in digital image, there is, for example, a technique of embedding the special information as a water mark in the digital image at the least influenced portion to the image quality in the pixel data (for example, LSB (Least Significant Bit)). This kind of the conventional invisible water mark data embedding technique is disclosed in, for example, Japanese Patent Publication Laid-Open (Kokai) No. Heisei 6-339110, "Image Information Transfer Method, Image Information Recording Device and Image Information Reproducing Device" (hereinafter, referred to as the conventional technique 3). The water mark data inserting method according to the conventional technique 3 transmits the image signal together with the copyright information and the generation information overlapping each other, to the space other than the valid image space displayed on a screen of the image signal, and performs the generation restriction of the copy according to the copyright information and the generation information included in the received image signal at a receiver side.
The invisible water mark data embedding technique, however, is defective in that only the water mark data is easily removed from the image data with water mark data embedded therein, without deteriorating the quality of the original image data. For example, the information corresponding to the LSB of the pixel data will be lost by use of a low pass filter. Generally, the image compression processing aims to reduce the data amount on the whole by diminishing the information amount with respect to the least influenced portion to the quality in the pixel data. Therefore, the water mark data embedded in the least influenced portion to the quality in the pixel data will be lost in the image compression processing. As mentioned above, the invisible water mark embedding technique has such a defect that it may be difficult to re-detect the water mark data in some cases.
Therefore, a technique of embedding invisible water mark data in a digital image as well as diffusing the water mark data into the frequency spectrum after frequency conversion of the image data (hereinafter, referred to as the conventional technique 4) is presented (refer to Nikkei Electronics p.13 (no. 660) 4.22.1996). Since water mark data is embedded in the frequency component of the image data to be processed according to the conventional technique 4, the water mark data is robust against the image compression processing and the image processing such as filtering, and the water mark data won't be lost. Further, random numbers according to the normal distribution are used as the water mark data, thereby preventing the interference of the respective water mark data even in the case of embedding a plurality of water mark data. Therefore, according to the conventional technique 4, it is difficult to destroy only the water mark data without much influence to the whole image data.
The conventional technique 4 will be, hereinafter, described with reference to FIG. 12. The water mark encode system according to the conventional technique 4 comprises, for example, discrete cosine transform (DCT) means 1210, water mark data output means 1230 storing the water mark data 1231, a water mark data inserting device 1240, and inverse discrete cosine transform (inverse DCT) means 1250. The DCT is adopted here as only one example of spectrally resolving means of original image data, and any other conversion means than the DCT will do.
In thus-constituted conventional technique 4, original image data is converted into frequency components by DCT, and n piece of data indicating higher frequency value are selected, each defined as f(1), f(2), . . . , f(n). On the other hand, each water mark data w(1), w(2), . . . , w(n) are selected from the normal distribution having a mean value 0 and a variance 1, and F(i)=f(i)+.alpha..vertline.f(i).vertline.*w(i) is calculated with respect to each i (where i=1, 2, . . . , n: hereinafter in the same way). Where, .alpha. is a scaling element. At the end, the image data with the water mark data embedded therein can be obtained as the frequency components by the replacement of f(i) with F(i).
The water mark data detection according to the conventional technique 4 is performed in the following method, by way of example. The original image data and the water mark data candidate w(i) must be known values in the detecting method of the conventional technique 4. At first, the image data with the water mark data embedded therein is converted into the frequency components by use of DCT or the like, the element values corresponding to f(1), f(2), . . . , f(n) having the water mark data embedded therein in the frequency band are defined as F(1), F(2), . . . , F(n). By the use of f(i) and F(i), the water mark data W(i) is extracted, calculated by W(i)=(F(i)-f(i))/f(i). Next, the statistical similarity C of w(i) and W(i) is calculated by use of the inner product of vector by C=W*w/(WD*wD). Where, W=(W(1), W(2), , W(n)), and w=(w(1), w(2), . . . ,w(n)), WD is the absolute value of the vector W, and wD is the absolute value of the vector w. As the result of the above calculation, when the statistical similarity C is a particular constant value or the more, it can be judged that the above-mentioned water mark data candidate has been embedded in the image data.
If creating the image data with the water mark embedded therein by embedding the water mark data in the original image data by use of the conventional technique 4 as mentioned above, it is effective for a writer owing the original image data to judge the illegality of the digital image data which may be illegal copy.
The conventional technique 4 requires original image data and water mark data candidate w(i) in order to detect the water mark data as mentioned above. Therefore, it is effective for a writer owing the original image to detect any illegally-copied image data, however, a reproducing device of a terminal used by a general user, because of having no original image, cannot perform the detecting processing of the water mark data. Then, a further improved technique than the conventional technique 4 for terminal processing, especially MPEG system (hereinafter, referred to as the conventional technique 5) is proposed.
In the conventional technique 5, original image is divided into blocks of 8 pixels.times.8 pixels and the water mark data embedding and detection is performed by use of the block unit for processing. In the water mark data embedding processing, the data is defined as f(1), f(2), . . . , f(n) sequentially from the lowest frequency component of AC component in the frequency band after discrete cosine transform in the MPEG encode processing, the water mark data w(1), w(2), . . . , w(n) are selected from the normal distribution having a mean value 0 and a variance 1, and F(i)=f(i)+.alpha..times.avg (f(i)).times.w(i) is calculated with respect to each i. Where, .alpha. is a scaling element and avg (f(i)) is a partial average obtained by averaging the absolute values of three points around f(i), for example, f(i-1), f(i), and f(i+1). The processing later than the MPEG encode processing will be performed by replacing f(i) with F(i).
On the other hand, detection of the water mark data will be performed in the following method. This detecting method requires no original image data and that only the data candidate w(i) (where, i=1, 2, . . . , n) must be a known value. In the block frequency band of blocks after inverse quantization of the MPEG decoding processing, the data from the lowest frequency component is sequentially defined as F(1), F(2), . . . , F(n). The average of the absolute values of three points around F(i) is defined as the partial average avg (F(i)), the water mark data W(i) is calculated by W(i)=F(i)/avg (F(i)), and the total WF(i) of W(i) for one image is calculated for every i. By use of the inner product of vectors, the statistical similarity of w(i) and WF(i) is calculated by C=WF.times.w/(WFD.times.wD). Where, W=(WF(1), WF(2), . . . , WF(n)), w=(w(1), w(2), . . . , w(n)), WFD is the absolute value of the vector WF, and wD is the absolute value of the vector w. When the statistical similarity C is a particular constant value or the more, it can be judged that the water mark data has been embedded in the image data.
Insertion of a plurality of water mark data into one original image data can be adopted in order to attach a plurality of information such as writer information and copy protected information, to one image data. The conventional techniques 4 and 5, however, is defective in making the circuit size larger because of inserting a plurality of water mark data in one original image data and increasing the processing procedure. Namely, the conventional techniques 4 and 5 disclose a technique of inserting one water mark data in one original image data, however, with no consideration taken to the case of inserting a plurality of water mark data (for example, two water mark data) in one original image data. Therefore, when inserting, for example, two water mark data therein, different water mark data 1231 and 1232 must be separately inserted by two water mark data inserting devices 1241 and 1242, or two water mark data must be inserted by letting them pass one water mark data inserting device twice.
Another example of the conventional technique of embedding water mark in a digital image is disclosed in Japanese Patent Publication Laid-Open (Kokai) No. Heisei 6-315131, "Information Embedding Device And Reproducing Device" (hereinafter, referred to as the conventional technique 6). The conventional technique 6 detects an area having no deterioration of the image even if replacing the data at the peripheral area when reproducing the information, for example, an equal background portion, by use of the relationship between a series of frames, and converts the level of the area to be converted, so to embed particular information therein. When reproducing the information, the area with the identification data embedded therein is specified by use of the signal lacked portion and the conversion information, and corrected so as to reconstruct the image.
The conventional technique 6, however, cannot embed the water mark information in all the frames, so that the frame with no water mark embedded therein is unable to protect against illegal copying. Since this technique is on the assumption that successive frames are of freeze-frame pictures and that there is no change on the successive frames, an area for embedding the water mark data cannot be specified in the moving images of violence action, thereby making it impossible to embed the water mark data therein.
Further another example of the conventional technique of embedding water mark in a digital image is disclosed in, for example, Japanese Patent Publication Laid-Open (Kokai) No. Heisei 5-30466, "Image Signal Recording Device and Signal Recording Medium" (hereinafter, referred to as the conventional technique 7). The conventional technique 7 converts image signals by frequency and embeds the information having the frequency signal lower than the frequency band of the image signal after frequency conversion. By the use of a high pass filter, the original image is taken out and the identification data embedded therein is taken out by the use of a low pass filter.
However, since the conventional technique 7 embeds the water mark data in the lower frequency portion in the frequency band after frequency conversion of the image data, the water mark data can be easily removed by use of a high pass filter. When embedding the water mark data in the intensive frequency portion after frequency conversion, a filter cannot remove the water mark, but when inserting a plurality of water mark data, it cannot help deteriorating the image quality.